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JOHNSON et al ON QUALITY EDUCATION: A CRITIQUE* 

byG.A. Welty and R. V/’ilkes 

The problem of substantiation of claims about quality education 
for all Americans has occupied a multitude of authors' for some time. 
The merit of their attempts to ascertain the truth concerning such 
issues as the bearing of integration, status, class, etc. on a dependent 
variable such as social adjustment or academic achievement varies 
widely. 'We do well to bear in mind the injunction of the great Italian 

social scientist, Pareto:. 

In a country where almost all the men are blond- 
haired, the majority of criminals will consist of 
blond men. But from that fact, one is unable to 
, draw a conclusion as to a relationship between 

the color of the hair and criminality. 

In an area of such ambiguity as the social studies, and especially in 

that emotion- laden subdivision of race relations, we cannot be too 

cautious about the quality of our research. This is not to suggest that 



5}C 

We would like to thank Mr. Alex Ajay, who drew our 

attention to "Quality Education and Integration" when it was first pre- 
sented, virtually the same, by Johnson in the summer of 1966 in 
Washington, D. C. Although Mr. Ajay severely criticised the Study then, 
and we have followed his criticism in some points (particularly the sta- 
tistical) of our critique, he is not at all responsible for any errors in its 

present form. 



p. 145. 



^V. Pareto, Les Systemes Socialistes , Vol. II, (1903), 



we should be in the least reticent about documentation of hypotheses, 
for factual evidence is sorely needed. Rather it is to suggest that we 
must be even more critical than usual of studies in this area, both our 
own. r 6 S© 3 .rch. sind that of oth©rs» 

«« Th© Study " 

Two focal points ar© drawn in th© following criticism of th© study 
contribut©d by Johnson et_aJL; (1) the research methods employed and, (2) 
th© ostensible validity of th© hypotheses which were advanced. Du© to 
th© apparent lack of rigorous application of statistical and experimental 
methods it may well be that Johnson etal. did not intend to imply 
the existence of any such procedure. However, "the Study" does con- 
tain some words, e.g., curvilinear relationship, which might tempt 
the unwary consumer to expect a degree of universality in the results, 
which is not justified. This possibility somewhat bolsters an apology 
for adding to the discouraging plethora of technical discussions 
found in literature. We are aware of and in sympathy with the rising 
concensus that there are too many approaches and too few results in 
educational research. In this vein, the authors trust that the reader 
too will find the second focal point of most iniierest. 

2 

: Johnson, Wyer, and Gilbert, "Quality Education and 

Integration: An Exploratory Study", Phylpn XXYIH (1967), 221-229. 

We refer to this as "the Study" throughout. All bracketed page num- 
bers in the text apply to "the Study. " 
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In "the Study”, Johnson et al have forwarded the theses that (a) 
the degree of racial conflict in integrated schools may be greater, aid 

f 

therefore the quality of education lower than in less integrated schools, 
and (b) in schools where the social class orientation militates against 
effective student -teacher relationships, the quality of education may be 
poorer than in schools where the social class orientation is facilitating. 

(p. 222) 

These two hypotheses suggest that racial conflict and social class 
orientation are being proposed as independent variables,' and are sug- 
gested as having some relationship to the dependent variable, college 
grade point average, (p. 223, note 3) The authors propose that . 

” this paper is a preliminary attempt to 
analyze the validity of these arguments". 

(p. 222) 

The words "preliminary" and "exploratory" apparently assume great 
importance in "the Study". Fortunately, one can ask "How much weight 

can they bear? " 

Before examining the "analysis" of Johnson et al , it may be of 
interest to examine their major assumption, namely that "the quality 
of education that students receive in high school is undoubtedly influ- 
enced by the quality of instruction" (p. 221). Generally, the literature 
does not support this supposition, i.e, consistent or systematic rela- 
tionships between quality of instruction and quality of education have 
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not been demonstrated. For example, forty years of literature abounds 
in studies which have failed to demonstrate systematic relationships 

between teacher ratings, i.e. quality of instruction, and student per- 

3 

formance or quality of education. 

One other preliminary point: Johnson et . al mentions that it is a 

"typical assumption that the quality of education is related negatively to 
the percentage of Negroes attending the school" (p. 223), We might 
remark that to make this assumption would be to commit just the error 

4 

Pareto warned against. 



D.M. Medley and H.E. Mitzel "Some Behavioral Corre- 
lates of Teacher Effectiveness," Journal Educa t ional Psycholggx 
Vol L (1950): also A.S. Barr et al , "Wisconsin Studies of the Measure- 
ment and Prediction of Teacher Effectiveness, " Journal Experiments 
Education Vol . XXX (1961), 5 ff. 



"The proportion white in a school was positively related to 
individual performance. This effect appeared to be less than, and 
largely accounted for by, other characteristics of the student body than 
the racial composition of the school per se. " Coleman, op. cit. _ PP- 
330-331. of. alsoR.E. Harriott & N.H. St. John, Social Class and ffie. 

Urban School (1966), chs. 2 &c 10, 



The Literature 



A synopsis of the literature^ would seem to indicate that (a) the 
school, as an institution, has little effect on the students' performance, 
when compared with the effect his out-of-school environment has on his 
performance,^ (b) one of the major academic (as well as social) benefits 

of the school is derived from heterogeneity of student body, i. e. , the 

i 



~ cf. The bibliography prepared by H. A. Aurbach, ’’Sel- 

ected Bibliography on Socio-Culturally Disadvantaged Children and 
Youth,” University of Pittsburgh; LRDC (1966), esp. 23-26, the ^ gal 
literature cited in Hobson v. Hansen , Federal Supplemet^, Vol. CCLXIX 
(1967), 201-519, esp. 493 ff, and L. Fischer andD.G. Lahr ’’Social 
Policy and Education” Rev. Ed. Research Vol. XXXVII (1967) 34-39. 



U.S. Commission on Civil Rights, Racial Is olation in the 
Public Schools (1967) Vol. I. pp. 81-82; Vol. H, p. 40 and Tables cited. 
Coleman et al have found a negative relationship between socioeconomic 
status andl^formance. This seems to be in keeping with the literature. 

op. cit s 309-310. 
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“interaction effect” of student on student, and (c) to the extent that 
variance in performance can be related to variance in the school (as 
distinct from environmental causes), the significant variance is between 
classrooms (i.e. intraschool). This is reflected in the debilitating 

g 

effects cf “homogeneous” grouping, track systems, etc. (Thus (b) 
is a corollary of (c). ) 

The preservation of some measure of openness both intraschool, 
by abolishing ’’homogeneous” grouping and inflexible tracking (if not 
tracking per se), and interschool, where the intraschool variance is 
negligable is of great importance. Negligible intraschool variance is 
generally the case. Heterogeneity, not homogeneity must be the means 
to achieving quality education. 

In light of this, it is important to see if Johnson et al can raise 
evidence against the general arguments posed in the literature. 



'^Cf . McLaurin v. Oklahoma State Regents ., United Sta^ 
Supreme Court Reports , (1950) p. 641 

On ’’tracking, ” see Hobson v . Hansen, op. cit. , pp, 
442-492. Homogeneous grouping is assignment as a function of abilityj 
"homogeneous” is as a function of race. On intra- and inter- school 
variance, see also Coleman, op. cit. , p. 297. 



% 



The Sample 

9 ' 

■We might wonder why only 19 of the 40 to 50 public high schools 

• 

in Chicago are represented in "the Study" and what happened to the stu- 
dents from the omitted schools. We might also wonder how representa- 
tive is a sample showing only 4 percent of the Negro students above the 
probationary threshold. Our doubts, based on intuition, can be sub- 
stantiated. 

• # 

An estimate of the variance of the author’s data, based on the ex- 

10 . 

ternal variance of the 19 a^d 15 samples, gives us 

tfh = 0. 88 grade points 
C*w = grade points 

The range encompassing all 15 means of the sample of white students 
is 0.70 grade points, the similar range for the Negro students is 0.71 
grade points. A chi-square test of significance indicates the prob- 
ability of the Negro sample deviating from a normal distribution is 
greater than 98.5%; the probability of the white sample deviating is 

greater than 99%» 

\ 

9 

The total number depends on how one classifies. Cf. 
R.J. Havighurst, Public Schools of Chicago (1964) pp. 203ff. 

10 

On estimation of variance, cf. E.S. Pearson ana 
H. O. Hartley, Biometrika Tables for Statisticians , Vol. I (1966), 
pp. 37-38. 

That is, 100% of the observations fall within 2. 39-3. 79 
and 1,77-3.19 grade points, respectively. Expected values on a chi- 
square test are 5 white and 11 Negro values. 
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We can then reach one of two conclusions! first, we can conclude, 
with an exceedingly high degree of confidence, that there were sy sterna- 

I 

tic biases In the selection of the subjects. If this is the case, no generali 
zations whatever can be made from the data. Hence, ’’the Study” is of 
no relevance in attempting to refute the statements in the literature, 
and hardly ’’preliminary. ” On the other hand we can conclude that 
Johnson et_a^ were not studying students at all, but rather high s chools. 

If this is the cas^, then the use of mean GPA is quite misleading. An 
index of the quality of education per high school is required, and the 
GPA of an assortment of students from a high school, ranging from 
one to one hundred and one students, simply won’t stiffice. 

The Hypothesis of Curvi linearity 

Johnson et al have indicated that a ’’curvilinear relationship” 
exists ’’between the percentage of Negroes in the high schools and the 
college grade of students” (p. 224) To indicate the arbitrariness,, of 
Johnson’^ assumption that there is a significant concave curvilinear 
relationship between racial balance and performance (i.e. the greater 
the segregation, the higher the performance), we will demonstrate 
that the opposite argument can also be asserted, on the authors’ own 
evidence. 

X»et us hypothesize that the relationship between racial balance 
and performance is convexly curvilinear, i.e. the dependent variable 



has its greatest value where the greatest racial balance is present. 
Given the data in Table I of "the Study" (p. 222), we fit two curves to 




Negro: = 0. 139: accept H 0. 99 5 

For the white students, there was no significant difference 
between the curve and the data, at a high level of confidence. The 
same was found for the Negro students. These findings indicate that 
practically any hypothesis asserting curvilinearity, with the curve 
flexed in either an upward or downward direction, can be supported 
by the data. Thus to choose any particular hypothesis as representing 
the "findings, " is manifestly bad practice. 
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The Arpiuments of "the Study'^ 

In light of this, it is perhaps not necessary to go further. With 
no data and hence no significance, little remains of "the Study" except 
some questionable political recommendations. However, we c^ ex- 
amine the internal structure of the "arguments" Johnson etal forward 
We must compare them to the literature, as there is no data in "the 



Study” to relate to them. 

On closer analysis, we see that racial conflict and class orienta- 
tion are not dependent variables at all! In fact, for Johnson 
racial integration per se is the only variable, hence must the indepen- 
dent variable. It is merely assumed by the authors that racial inte- 
gration is concommitant to racial conflict. With regard to the data 
presented in "the Study" (again, assuming contrary to fact, that it is 
meaningful), the only argument that could be made is that integration 
has a debilitating effect upon academic performance. 

Johnson et al state; 

"Students probably experience considerable conflict 
in many schools undergoing transition. This conflict 
may be manifested in the relationship of students 
with one another. . .the degree of racial conflict in 
integrated schools may be greater, and therefore the 
quality of education may be poorer than in schools 
that are either predominantly Negro or predominantly 

white (p. 222), 



We could symbolically express this hypothesis (overlooking the modality 



of "may" , with I as integration, C as confUct, and E as quality of education) 

(+I — »-vC) — *(+I — » -E) 



as 
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Thus we see that integration (I) is in fact the independent variable, 
rather than racial conflict. Thus we see that one of the "two factors 
which seem particularly important" (p. 221) is not important enough 
to be an independent variable ! 

We find that the best description provided in the literature of 
the actual state of affairs is symbolized by 
' +I > +E 

This is supported by the available evidence, and holds irrespective 

12 

of the relationship between integration and social tension. Johnson 
et^have merely suggested a false hypothesis, supposing an inverse 
rather than direct relationship to exist between integration, decreased 
racial tension, and quality education. And, of course, with no data, 
Johnson et al had no means of testing the hypothesis independent of 

the literature. 

The Orientation- Performance Hypothesis, 

Furthermore, it is impossible to ascertain whether the ascription 

of social class orientation was in fact an independent variable (i.e. that 

there was a measurement of this variable logically prior to that of the 

12 , , . 

Cf. note 6 above. As long ago as 1948, E.S. Bogardus 

showed, in small-group experiments, that integration decrease j. racial 

tension. Cf. his "Reducing Racial Tension, " Sociology. and Social 

Research, Vol. XXXV (1950), 50 ff. and literature cited there. 
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dependent variable). If this measurement was prior, no indication 
of the significance of this variable can be ascertained, in its relation 



to college grade point average. 

As a matter of philosophy or practice, when one fails to demon- 
strate an anticipated linear relationship in a sample of data, it would 
seem more sound to question the sampling procedures involved rather 
thkn the "true" nature of the phenomenon under investigation by intro- 
ducing another theory ad hoc . That is, in view of the rather large 
number of previous investigations which have demonstrated a negative 
Hnear relationship, it seems rather presumptions to ascribe the 
deviant results of one investigation to the phenomena, rather than to 



a faulty method. 

Further, we have seen that the claim of cur vi linearity cannot 
be substantiated with respect to Johnson's data; particularly the 



13 ... 

It is doubtful: the authors report (p. 227, note 5: the 

asterisk in Figure 1, P. 223, refers to this note, not footnote 3) that 
the unclassified school " probably has a middle -class orientation (italics 
ours). Either their taxonomy violates the necessary condition of com- 
pleteness, or else was constructed ad hoc to explain away variance. 
Neither is legitimate. Consider also ". . .as an isolated category the 
concept of class is meaningless even for purely descriptive purposes. 
..."class" is not merely a defined term but a category imbedded in a 
theory. Concept and theory of class are inseparably connected." 

R. Dahrendorf Class and Class Conflict in Industrial Soci e^ (1959) 
p„ 152. The assumption that "social class orientation" is meaningful 
in this theory-free context is clearly at discord here. 



apparent absence of a linear relationship does not establish that there 
is a systematic curvilinear relationship between integration and perJ 

I 

formance. With reference to Fig. 1 (p. 223), Groups (or ’’Settings")'^ 
C & D do not appear to show any systematic relationship between 
independent and dependent variables. On the other hand, groups A 
& B visually appear to demonstrate the anticipated linear (negative) 
relationship. Visual inspection of Fig. 2 (p. 224) does not appear to 
be anything other than a negative linear relationship. In any event 
visual inspection is not the best means available to confirm or to 
deny the existence of systematic relationships, especially where 
data have been grouped, but is generally recognized as a heuristic 
approach for the trained statistician to ascertain the need for further 
(non- intuitive) tests. In very elementary lectures in introductory 
statistics, students are enlightened to the fallacies encountered or 
perpetrated with grouped (arbitrarily or not) data. To be more pre- 
cise, a trained statistician would hardly artificially decrease the 
range of one of the correlates (i. e. GPA) when attempting to demon- 
strate either a linear or curvilinear relationship. 

In relation to the quantification of the independent variable, if 
the dependent variable is social class orientation, then the metric 
of social class orientation should appear on the abscissa, not per- 
centage of Negro enrollment. If the percentage of Negro enrollment 
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and the index of social class orientation are so well correlated as to 
allow the type of substitution proposed by the authors, then it should be 
demonstrated, Johnson, et al , have simply failed to show on what basis 
(if any) they have quantified social class orientation, 

A problem related to the dependent variable (grade point average) 
is again the problem of restricted range. One might anticipate that any 

i . / 

Hnear curve representing grade point average might tend to curve (appear 

asymptotic) due to the fact that few grade point averages approach either 

extreme of the continuum. In this case a change in slope of the curve at 

either extreme can hardly be attributed to the other variable (social 
14 

orientation). In the absence of measures of variance within groups, 
however, it is not even possible to know what the extremes would be. 

This fairly well concludes our analysis of Johnson et al as a 
serious contribution to the literature on integration and education. We 
have demonstrated that sample defects and lack of statistical significance 
reduce their data to nil. Their hypotheses appear a shambles of ques- 
tionable asstimptions. In conclusion we would like to pose some prob- 
lems which we hope can be treated in the near future. Perhaps we could 
label these our "Suggestions for Action," 

14 

C£. F.Y. Edgeworth, cited in Pareto, Ecrits sur la courbe 
de la repartition de la riche sse (1965), p. xvii. If a curve "relates to 
averages it is not to be expected that it should be verified at the higher 
extremity where only one or two observations occur," 
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C one lu sioTi; Needed Research 



Questions which can be posed here are perhaps more far reaching 

» 

than Johnson et al would care to address themselves to, Ihe findings pf 
aubstantial studies of the Chicago public schools indicate that the usual 
negatively sloped relationship, pairing predominantly white schools 
with higher academic performance of students, and predominantly non- 
white schools with lower performance, exists there, as it does through- 

15 

out the Northeast United States. 

Both the University of Illinois and the Chicago public school sys- 
tern are,to some extent, state -supported institutions. As both are at 
least partially maintained by the State of IlUnois, an interesting legal 
possibiUty arises. In the face of differential interschool allocation of 
elementary and secondary school resources, where the school has been 
accredited by the appropriate State Agency, would the State institution 
of higher learning be obliged to estabUsh corresponding differential 



entr 



r 



nee re 



quirements, probationary thresholds, etc. 



? 



15 

Cf. J. Burkhead et al , Input and O utput in Large-Cit^ 

High Schools (1967), Ch, III "Chicago Public High Schools, " esp. Table I, 
P.47) showing a negative correlation of 0,79 between percentage non- 
white and median family income, and Table III (p. 51), showing a nega ive 
correlation of 0, 69 between median family income and school dropout, 
and Tables IV and V (pp. 52-53), where a positive 

0. 91 is found between Uth grade I.Q. , and 11th grade Davis Reading Test 
scores, and median family income, respectively. 



This would seem only just: it appears rather vicious to suppose 
that there is somethiug good about beiug a Negro student from a pre- 
domineutly Negro high school, when this indicates one is in the greatest 
danger of academic failure as a result. This can be illustrated by the 
data of "the Study".A 2 by 2 table for variables "student's race" and 
"racial composition of school" gives us 



Student’s Race 
N W 



School N 

Composition W 



2.45 


2.78 


2.751 


3.40 



This indicates (if the data mean anything. at all) that the white student 

from a predominantly white high school will perform almost one grade 

« 

point above the Negro student from the predominantly Negro high school. 
Although mentioned in passing in "the Study" (pp. 223-224). this was 
apparently overlooked when it is proposed that "the integration issue 
aside. Negro students would receive a better academic education" (p. 228). 
Of course, we cannot generalise from the data, so we properly confine 
our Vemarks to the pUght of the Negro students reported by Johnson 

et al . 

A proposition was recently forwarded that "a university must 
be held accountable for the quaUty of education in the school system 
in its geographic home". In view of this, assuming equal per pupil 

Urban Education, Vol. Ill (1967), No, 1, p. 6, 
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expenditures, which incidentally would appear to be the case for Chicago, 
is the institution of higher learning obliged to provide preferential accomo- 
dation (as above), if the public elementary and secondary schools have-not 
made the differential allocation of resources (such as busing) which would 

permit equality of educational opportunity? 

The stipulation that "The school must not lose sight of its function 

and must not compromise its obligation to strive for maximum educational 

development for all children in the community" is interesting. If true, 

one wonders whether the school board and the school administrators are 

legally responsible for the miseducation of Negro children in what 

' EUas Blake has called "criminally neglected schools, " in any sense of- 

responsibility other than that of Carl Hansen, (ex) Superintendent of 

. . 

Schools in the District of Columbia? 
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As reported in Burkhead, op. cit . , Table X, p. 57, where 
expenditures are given as $477 per pupil for low. and $459 
high'median family income. The coefficient of variation was 15%.. 



18 



Urban Education, op. cit. , p* 7. 



19 

Aa fouud in Hobson v. Hansen , op. cit . , pp.418-419. With 
O.M. Fiss, "Racial Imbalance in the Public Schools. " Harvard 1^ 
Review Vol, 78 (1965), 584, we are suggesting the label de fac^ (or 
SiS U .M.M for .ohool ...h o.m.. .0 .v .-O 

their responsibilities. Blake is cited from the Harvard Educatio nal 
Review , Vol. 37 (1967), p. 473, 
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Have all the means to integration been sincerely tested? We 
wonder if Johnson et al are naive enough to think ’’the current integration 

f 

policy of transporting Negroes to white schools" (p. 227) constitutes a . 
comprehensive busing plan. As a matter of simple logic, one doubts 
this. With the exception of radical discrepancies in pupil per classroom 
distribution,^® the movement of large number of students in one direction 
alone is impossible. Referring to "large-scale busing," Fiss remarks 
that "Such a plan would require transportation for both white and Negro 
children. We also wonder if Johnson et al , in suggesting "the white 
exodus. . .adds further complications to the issue of racial integration 



(p. 228), might not be overlooking that 

"There are many methods of pursuing the goal of integration. . . 
(the) possibility of massive withdrawal, which varies with the 
intensity of the whites’ resentment, may be affected by the 
costs of private schools and by the ties of family may have 
with the city. Although (busing) is within the spectrum of 
those capable of correcting the imbalance, it should not 
obscure the fact that less extreme remedies exist and that 
in most communities they would not only alleviate the im- 
^ balance, but eliminate it as well. 

20 

As is the case in Washington; cf. Hobson v. Hansen, jo£, 
cit. pp. 433-434; not so, apparently for Chicago: cf. Burkhead, op.cit* > 
■^58, Table X. 

21 

O. M. Fiss, op. cit . p. 573. 



Fiss, op. cit. p. 573-574. This would seem to relate to 
Joseph Alsop’s comments as well. Cf. his "No More Nonsense about 
Ghetto Education!" New Republic, 22 July 1967, pp. 18 ff. 



Answers to questions such as these would seem to be the product 

of true exploratory studies. After all, by the best indicators we have, 

it appears to be the case that to upgrade school facilities and programs, 

to increase the number of teachers and decrease class size, etc., while 

perhaps improving the "quality of instruction" would have Uttle if any 

23 

appeciable bearing on academic performance. What it means to 
undertake "exploratory" or "preliminary" studies, in an apparent 
ignorance of the (really) voluminous literature, it not at all clear. 
Given the recommendations for action of Jo hns on et_al, it does 
not appear to be the case that "Negro students would receive a 
better academic education" (p. 228), They will continue to go to 
college to play basketball and flunk out. In conclusion, we call 
for a little more vision, a little more scope, and a lot better 
research. 



23 

■ Coleman, op. cit , , pp. 22-23; 293-302. Also Burkhead, 
op, cit . , p. 56: "He who would explain variations of Chicago public high 
schools has a great deal to interpret. . . socioeconomic variables are of 
the greatest importance in determining differences in school outputs. 

The out-of-school variables are far more significant than the in-school 
variables. " 
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